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SUMMARY 

Problem v v. » 

" * -2 • ^ . ' 

With the rapid growth in the volume and the importance of technical documentation, 
it is becoming increasingly important that technical and training manuals be easy .to use.. 
An important factor in determining ease of use, which has been especially difficult, to 
effectively describe or control, has been the - comprehensibility of the materials. v 

' , .ft 

, background "v . « , 

• „•••*•■ 
tJ- >A11 of the services have turned increasingly to the use of readability formulas to 
guide and regulate comprehensibility of prose in the production of technfcal documents. 
'The readability formula is an empirically-based method for predicting comprehension of a 
text from a count of certain of its word and sentence characteristics. These formulas are 
used as guidelines or feedback to' ^e, writer, identifying the expected comprehension 
difficulty of a draft, and as criteria that the writer imust achieve. Writers are not 
expected to achieve the readability criterion simply by writing to the .formula (e.g., by 

.using a word with fewer syllables regardless of whether or not it is a "better"' word). 

; "Rather, they are expected to follow readable writing- guidelines (e.g., use more familiar 
. words and direct simple sentences). This application of the readability formula presumes 
a strong causal relationship between- readability and comprehension and readable "Writing 
techniques and comprehension. However, there is little^support for these * causal 
assumptions. , t 4 • 

• ■ ^ * o 

Purpose • • /' * H " * ' .. \ x 

* ' T K h .?. P urpose P f this research was to determine tiie extent to which increasing 
readability by carrying out readable writing revisiohs will improve comprehension, v 

. Method *. . „ 

» • i - - -'•*.. 

- " Ei §ht passages from a reading test were revised by using word lists .to simplify the 
'Vocabulary and a restriction in syntactic structure to simplify the sentences... Vocabulary 
simplification reduced the readability formula score from a reading gr,ade level (RGL) of 
11.3 to 9.6, while sentence simplification resulted in an average 7.7-RGl!'for the passage's. 
When both words and* sentences were simplified, the readability was reduced,, to 5.5 
RGL— a reduction of six RGLs from the original materials. ' 4 ' 

These passages were . pVesen ted to Navy recruits in a series of five experiments in 
which the reading task (reading-to-do vs. reading-to-learn), -the time* allowed for reading,' 
and the comprehension tests (multiple-choice' vs. cloze) were. varied. 

^Findings - ' * • • ■ ' \ 

- Except 'for one instance, the- "readable writing" revisions had no practical effect on 
comprehension regardless of the reading skill's -of the recruits. The one positive instance ' 
of practical ^ignifioince>as for low ability readers when the reading task was^ in a. 
i reading-to-learn formatTXe., the reader did npt know, what questions would be asked untii 
the material had been read, and 'removed,). Even here, however, onl : y vocabulary 
simplifications— the text manipulation that resulted Jn the smallest change in the 
readability score —produced a significant effect.. ' •'.*"• 



inclusions 



The results indicate that "readable writing" revisions can facilitate comprehension 
under very particular circumstances. However, the resulting changes in the readability 
score' §re not in any way predictive of the changes in comprehension. Thus, a ^formula 
score is neith^ an effective guideline nor aa effective criterion to impose orva writer. 

Recommendation - r 



The text production process should be studied to determine how both government and 
contractor management of the production effort can be modified to increase attention to 
comprehensibility. This should include a consideration of computer-aided authoring, 
management priorities, and personnel qualifications. The results of .this study fail to 
support the use of a readability formula as a means of controlling the comprehensibility of 
text. 

• 1 » 
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INTRODUCTION 



Background and;Problem - "'* . 

m Written text, whether it is in the trad^ionaK.bxDoklQf.m pr^s microfilm or computer 
display, plays-a central role throughout so4(^v^^ the primary 'means of communicating ^ 
and documenting information. The ' rapid^^Ertsion of technology with its associated* 
knowledge requirements has greatly increased bo,th-the volume and the importance of text 
documentation/ All facets of "society have entered the information ag£ and, as a } 
consequence, it is seldom the case today th^t "experts" in any field of endeavor can^ 
posfeess all the necessary knowledge "in their head." Instructional textbooks are growing' 
in number and in degree of specialization. Professional journals are dividing and 
multiplying. . - ± 

The incredible growth in the requirements for text documentation and the effects on 
the .individual can perhaps best be illustrated by the military's technical , manual % 
requirements. According, to estimates, the U.S. Navy alone has 25 million pages of* 
.technical manuals (not including training texts). and 400,000 pages are added or revised 
yearly (Sulit & Fuller, 1976). In the^lasf 25 years," the nucnber of pages required to 
document a moderh weapon system has increased by 14,000 percent (Muller, 1976). This 
growth in materials obviously has not been accompanied by a computable growth in the 
number of naval personnel. In fact, manning levels in some ship classes, including 
destroyers, have decreased (Aik.en, 1980). Thus, there is a tremendous increase in the 
documentation that any 6im individual must.utilize. * When this increasing requirement for 
information is added to t\m national decline in literacy skills of the users and the high 
employee turnover in the military as well aj in, industry, it is .obvio v us that information 
texts-must be designed fojHriaximum;Usability. - \ j s [ 

The usability of a- text can be described in terms of four dimensions: access, 
'accuracy, completeness, and comprehensibility. The reader must be 3^1e to locate the 
relevant information. On<j[e located, the information must be accurate and be described 
to a level of detail that matches the information requirement'and the knowfedge level of 
the reader. However, even^if the information is compdete, it may not be^alite if it is 
written in a convoluted, jargon-ridden fashion and 'is not Supported by adequate graphics. 
While all four dimensions are important to usability, the last dimension — comprehensi- 
*bility--has been the jriost troublesome and is currently receiving the greatest attention. 

v.- There has been Vn^tionwide cry for clear, comprehensible writing. The focus of this 
concern h^s been on fconsumer* documents and -on industrial and military manuals. These 
types otf documents pose a special problem because <hey ege not sold as separate Itejns 
but, ££ther, are given away or accompany some otfher, more major purchase. Hence, they 
are not subject to the typical marketplace controls fo^ producf quality: • The legal 
► doc^irnertt that comes with the insurance policy the agent described is simply accepted. 
^Similarly, if customers want a particular piece of equipment — be. it a modern toy, a 
'mixer, J>v 3 radar sevthey must accept the accompanying\assembly, use, and maintenance 
instructions. If they want the primary proddct, they sirhply accept the documentation. 
From the producer's point ot view, the customer is purchasing the primary product; thus, 
the features and the cost of the product are most important. Tfyere is little incentive to 
irhprove the quality of the documentation; indeed, there. ^ a disincentive sin£e such 
efforts would only add to the cost of the primary product. * I^P* 

It is obvious that, .under these conditions, there is little reason for the producers of 
these types of documents- to expend the time or the money to ensure, adequate 
comprehensibility. Although it can be demonstrate time and again that documents can be 
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redesigned to make them more comprehensible (see any issue of Simply Stated , Battison <5c 
Landesman, 1981), the difficulty is to get the publishers to produce 4 that type of -document 
under normal conditions and funding. If this is to be achieved with thbse-producing 
. documents for the Navy, then either some means of regulation or some type of incentive 
< system is required. An incentive system isianlikely since this will simply add to the cost. 
Regulations require some sort of assessment systini and, unfortunately, comprehensible 
writing is not readily described, prescribed, or assessed by objective rules and procedures. , 
Testing of personnel to -determine comprehensibility is typically too costly^ and not 
feasible logistically* Consider, for example, the requirements for testing personnel on the 
400,000 pages of technicaf manual text produced annually by the Navy alqne. The use of • 
judges to determine the comprehensibility of text ^wobld appeaTr r to be a reasonable 
alternative to direct testing. However, judges simply do not agree on their rdnking§ of 
comprehensibility (Carver* 1974; Wright 1 ).* Further, even .experienced writers *disagree 
and contradict eath* other in thfeir rules for comprehensible writing (Klare^ undated). 
Thus, it wpuld seem that comprehensible writing currently is largely an art not readily 
subject to objective description^ » . x 



s 



Despite the se difficulties, the requirement fo^af legally binding comprehensibility^ 

criterion remains. To meet this requirement^ijiany organizations have turned to the 
readability formula as a text production guideline and criterion. Pressman (1979) reports 
that two thirds of th^states requiring, clear writing in cdnsumer regulations have • 
specified a readability formula score as the criterion to be met. Similarly, nrfilitary 
specifications for the preparation^ of manuals are increasingly irfvoking a readability 
formula score to "fnatch the manual to the people" (Department of the Air force, 1977; 
.Department of the Army, 1978; Department of Defense, 1978, 1980).* .While the 
readability formula is an objective device, it is not based- on the specifications of 
particular writing rules. Furthermore, formulas are inexpensive and easy to apply. Thus, 
.it" would seem *that their application has potential as a primary technique for regulating 
the production of Comprehensible text. The readability formula, of course, only attends 
to the writing style of the prose. It doestjot address graphics, graphic-prose coordination, 
format, or organization.- Nonetheless, if a readability formula is effective as a criterion 
for the .prose, then at least a significant portipn of the battle to obtain cbmprehehsible 
Vext will have been won. v • * / • 

' While readability formulas are objective and easy to apply during text production, it 
i£ still not clear if their use as a criterion or as feedback: in the production procfess is 
valid.' These formulas have all, been developed as empifical tools to ^predict comprehen- 
sion of text that is^already written. No model or theory of comprehension underlies the. 
text variables used to predict comprehension. " They are simply the best predictors that . 
may or may ndt be causally associated with comprehension/ Yet* obviously, there is 'an 
implicit causal assumption underlying the application of a formula in regulating tlfe 
production of text. , - *'* 

* , ^Klare (1979) has argued that readability formulas can be effective aids in preparing 
more comprehensible text, as long as the writer does ncft write to the iormula. That is, 
Klar£ recognizes that the readability^ formula is a predictive tool and simply changing text 
to meet the requirement^ of a formula (i.e M by shortening words and sentences) will not 
necessarily result in more comprehensible text, even if a lower readability sdore is' 
achieved. ^Writing to the formula wolild only work if pne pre surped, that the particular 
predictor variables (e.g., number of letters per word)- were also the causal Variables^. 

: : " 4 * 

bright, P., personal communication, November 1980. 



While fclare (1979)- dS^s not advocate writing to the formula, he does advocate the. 
use oiyVeadable writing", techniques both to reduce the formula scgre and increase 
rcompreroensibility. A readable writing approach, while not necessarily addressing the 
formula\ variable, is focused exclusively on variables <at the word and sentence level 
(Klare, "1975^. "Thus, instead of mechanically using shorter words, .more familiar and 
*concrete\words. whicfi, in general, wiir'be' shorter, are sought. Thus} both a readable 
writing approach and writijig to formula result in shorter w<5rds*and sentences. They are' 
perfiaps bast distinguished by the attitude o£the writer--dqes he/she" search /or a " f goo.d" 
short wor-d\or simply a short word? - * 

* A readable writing approach is, in fact, tfye only, option (outside of writing to the, 
formula) opfrn to a writer fa revise the draft of a, text fp nrteet a readability criterion. 
Since the predictor variables are virtual!^ always at the "word and sentence level,' revision 
.rafust address warjd and sentence factors. Im^rovirtg the* format, reorganizing,^idding. 
xamples, ajiffl defining objectives are all revision techniques that might improve pdmpre- 
hension, but &fe irrelevant to a readable writing approach and would have- little effect on 
a readability ^eore* Thus, the use of readability feedback or a ceadability criterion, ^long 
with readable Writing techniques, presumes that the primary causal factors in compre* 
hension are aisThe word and sentence level. The\ fact that readability predictors correlate 
so strongly witn comprehension of already prepared text might lead one to accept such a 
presumption. Hpwever, it may be that a good writer is a conscientious writer; he/she not 
only uses simply words and sentences but also orgknizes well,* presents examples and 
transition information, etc. If -sonthen the use of a readability formula criterion will not*, 
lqad to significant improvements incomprehensibility even if readable waiting techniques 
"are used. \ " \ 

There is a wealth' of data, (e.g., Adam's, 1967) indicating the importance of word ancj 
sentence yariables- in learning and recall. .Because of these d&tay one might presume that 
readable writing techniques would be effective, if not wholly as effective as the 
readability formula multiple regression valu^ would suggest, rtowever, $s the^ cteta are 
'primarily from verbal ^earning studies, surprisingly 'few are from large text segments. 
Go£tz (1975), in a recent review, reports an "embarrassing" lack of empirical evidence on 
learning anef recall diff fences between sentence lists^ and text. Klare (1963) reportsx>rriy 
six readability, studies involving the controlled manipulation ok word or sentence diffi- 
culty. Orrly one of those six^studies found a positive effect and this was for simplifying m 
vocabulary.. In- other studies,rTuirrffian and Brady (197,3), rather than substituting more 
familter worlds to^simplify text, made unfamiliar vocabulary in the text more compre- 
hensible to the readers by giving extensive vocabulary training sessions.. While the. 
training led to & 2Q-percent improvement in vocabulary knowledge, there*was nb effect on. 
text comprehension. Cplfeman (1962), using a section of a psychology textbook, increased 
sentence length to an average of 38.6 words in one condition and. shortened sentences to 
an average of 15.4 wprds^in another condition. Although he used Flesch's (1948) guidelines 
for readable writing to revise the passages, this very large sentence length, manipulation 
resulted in only marginally significant differences in comprehension. " 

- -* » * ' ,' . 

Siegel, Lautman, and Surkett (1^74) simplified both the vocabulary and syntax of^ 
technical {materials following readable writing guidelines. While the^ sifoplificatiorr 
resulted in a 3.5 RGL improvement in readability,' there was a marginally significant 
effect on comprehension in only one of .three experiments? Kniffen, Stevenson, Klare^ 
Ehtin, Slaughter, and Hool<eJJ979) also failed to find any effect on comprehension when* 
they simplified technical materials, even when readability -was four grade levels less tfoan 
the RGL of the participants. . , * 

^ j 

c Klare (1976) presents a model of the' text use situation that places ljjaaits on the 
circumstahces under which readability improvement will result in h^her comprehension 1 . 



Klare proposes that manipulation in readability may be ineffective if the reader is highly 
motivated, familiar with the subject area, or has excessive reading time. In addition, he' 
suggests that improved comprehension requires the manipulation of both the sentence and 
the word difficulty variables as well as large differences in readability that bracket the 
skill level of the reader. From his review of readability^research, Klare (1976) conclude^ 
that Ynost null results in readability manipulation can be accounted for by one of these 

factors. # 

• » — 

Purpose 

* 

The present research nought to test the effectiveness of readability revisions. as a 
means of producing more comprehensible text. Revisions for readability were not written 
to formula directly but, rather, by using "readable writing" guidelines. The objective was 
to achieve large changes in readability of the passages by following strict readable writing 
guidelines, thus simulating what a writer wouldi try to do in/revising for readability. In 
addition! the purpose'was to test the effect of the readability manipulations uncier the 
most favorable conditions possible, thus maximizing the potential for a "readability 
effect." Ih line with Klare's (1976) model, jeaders.were unfamiliar with the topics and had 
low motivation. Further, their reading skill varied over a wide range so that the' 
interaction of reading Skill and readability could «be assessed. 



■GENERAL METHOD* 

Desigrvand -Materials / 

Eight passaged from the Nelson-Denny >Reading ^Test -Form B, Part II* (Reading 
Comprehension) (Nelson & Denny, 1960) provided* the. basic materials for all of the 
experiments. Four narrative and four expository passages are in the test*. The first 
passage consists of 597 words, while all of the others are approximately 200 words long. 
The Nelson-Denny test questions—all five-choice, multiple choice- -wer^ also used,\n most 
of the experiments. There were eight questions associated with the long passage; and 
four, with eac/i of the others. * 

Five experiments were conducted. In each, there were three* factors--one spkject- 
related and two passage-based. The vocabulary and the sentences of each passage* were 
manipulated independently to create the two independent passage factors- -sentence 
complexity and Vocabulary difficulty. There were two levels for each factor, original and 
simplified thus yielding four versiQns of each passage. The various versions for one of the 
passages is presented in the appendix. Ability, , the stibject-related factor in each, 
experiment, was assessed by performance on an alternate form (Forn> A) of the Nelson- 
Denny Reading Test (Nelson <5c D^nny, 1960). ^ / • 

For the vocabulary simplification condition (Condition V), the words^in all Nelson- 
Denny passages and* in tt\e multiple-choice questions had to meet one of the following 
criteria: (1) A or AA on the Thorndike-Lorge word-list (Thorndifte & Lot^e, 1944), .(2) no 
• higher than the foyrth grade level on the Durell (1956) word list, and (3) included in the 
Dale list of 400 words (Dale* & Chall, 1948). Twenty-six percent cff the content 'words 
failed to meet these criteria and were changed by substituting words from one of the lists. 
This amounted to changing 13 percent of the t6tal number of words in the passages. 
Excluding proper nouns, there were only eight* words that failed to meet the criteria and 
for which contextually appropriate substitutes could not be found. These words were 
unchanged. Substituting more frequent and easier wprds 'also resulted in shortening* the 
average word length. The mean number of syllables p^r word was 3.18»for words removed 
and 1.84 for words added, h 42-percent reduction in syllable length for changed wor.d's. 
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For sentence simplification (Condition S), the criterion that every sentence be m 
simple sentence Vith no morejhan two phrases was, applied to the passages and multiple- 
choice cfuestions. Pronouns, articles, and connectives were abided and deleted as 
necessary to meet the criterion and to maintain the continuity and flow of the stories. 
Applying this Syntactic criterion resulted in shortening the average sentence length from 
21.3 to 9.8 Words, a reduction af more than 50/percent. ; 

Finally, both the vocabulary and the sentence simplification criteria were applied to 
the passages and questions (Condition S-V). It is this join* manipulation of the readability 
factors that Klate (19/^6) considered essential for producing comprehension effects. 

* N 7 Tfie effects of these manipulations on readability scores were determined by applying 
three readability formulas to each passage in each of the experimental conditions as well 
as to the passages in their original form (Condition O). The formulas us$d were 'the 
Kiricaid revision of the FJesch formula for use with the military (Kincaid,>ishburne, 
^ Rogers, <5c Chissom, 1975), the' FOG count ^Gunning, ' 1952), and th$ Fry formula (Pry, 

.1968). As can be seen in Table 1, all three formulae clearly indicate that the 
simplifications significantly increased readability with the effect being greater for the 
sentence (Condition S) than for the Vocabulary (Condition V) manipulation., The 
distribution of Kincaid-Flesch, scores, between Conditions O and S-V <do not overlap % Thus, 
the manipulations were effective in producing large differences in readability as required 
if comprehension effects are to be obtained. ' ^* 
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Table 1 



Changes in Readability as a Function <?f Word and 
Sentence Modifications ^ 



Reading Grade Levels (RGLs) 



Revision ^ 
Condition * f - 


Words 
Per 
Sentence 
(Mean) • 


' Percent of 

Content 
Words Changed 


Kincaid- 
Flesch v 


* 

* FOG ' 
Count 


Fry a • 


Original (O) 


21.3 


0 


11.5 • 
8.5-16;6 


15:1 - 
. 8.0-21.4 


11.1 / 
7.0-College j 


Vocabulary (V) 

i 

i * 


22: 0 


26 ' 

• 


** 10.1 
6.7-14.8 


12.8 
7.9-17.8 


r ■ ' 

« 6.0-12.0 " 


Sentence (S) . 


9.8 


oV 


7.3 • 

4.2-11.2 


9.9 
-6.6-13.3 


9.3 
4.0-College 


S-V 


10.2 - 

r 


26 

e 


5.5 
3.4-8.2 


7.4 
5.3-9.6 


6.0 
3.0-9.0 



*Fry scores of "college level" were counted as 14.0 RGL for averaging. 
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Only four of the multiplp-choice questions required minor syntactic simplification to 
meet the criterion, irfn^t^rhe' syntax 1 of the questions was virtually the same for alj 
conditions. However, . significant changes in vocabulary were required to meet the • 
voc&bulary criterion./ The terminology ased in thej>as^ages"andTh "the questions was kept 
corisistent. Therefore, the vocabulary of the questions for Conditions V $nd S-V was 
different, from that under Conditions O and S ? • . „ 

Subjects and Testing Procedures , „ „ . 

\ The participants in. aU *the experiments were male Navy recruits approximately half- 
way through training atthfe Recruit Training Command, San Diego. Recruits at San Diego 
are typically between 17 and 20 years old and approximately 80 percent have at least a^ 

high schobl diplonha. „ : 

. '* - , . t • * 

Participants w^re„ tested in groups of 40 to 70 in- a large test hall having individual 
carrels/ Each session was monitored by three experimenters. A* session began with 
pretesting for'a&Mity using Part II (Comprehension) of Form A of the Nelson-Denny test 
(i.e.,* an alternate form of the basic experimental materials). The test >yas administered 
using the published test administration* procedures (Nelson Denny, 1960). The' 
experimehtal materials were then "distributed with all -experimental conditions equally 
represented. , * ^ * 

EXPERIMENT! > 

Sticht, F<*x, Hauke, and Zapf (1977) have distinquished between two types of reading 
tasks: reading-to-do and reading-to-learn. A reading-to-do task involves reading with .a 
specif ic, Viarrowly defined objective for immediate application; reading is a subtask of a 
•Specific job* task. In the reading-to-le^rn task, the objective is to store and retain what is 
read for use at another time. , ' 4 

In this experiment, the effects of t the readabUity*Hnanipulation were evaluated on, 
reading-to-do tasks. Sticht et al. (1977), in interviewing Navy personnel, found that 70 
percent of the reading tasks of job incumbents w'ere reading-to-da. Of course, if one 
considers* the use of technical manuals/ the tasks are almost exclusively reading-to-do.* 
Thus, if readability formulas £re to" be effective production tools for the military, 
readable writing techniques must improve comprehension in r,eading-to-do tasks. 



— «? — Irrthe reacfing-to-do task for Experiment l r the ^participants were givefh the oppor- 
^tii\i*y to' study the four or eightTquesti ons on th$ passages before' reading the passage. 
After reading was completed, . the questions were presented again for the participants /to 

* answer,. ^ Thus, memory requirements were minimized and the reader knew the specific 

* information objective before^eading the passage. k , \ 

M ethod " i. . < 
■ - 

"A total o#230 recruits were tested, with 54 to 60 assigned to each of the four 
conditions. A^ter pretesting, each participant received. one of the experimental sets of 
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booklets. In each booklet, materials for each passage were sequenced on successive pages . 
asf follows: the multiple-choice questions, a blank page, the passage, a blank page, and the • 
multiple-chbice questions. ¥he order of the questions and the alternatives within a - 
question was randomized from first to second presentation. 

• ? - * 

Participants were told that this was a reading test, like the pretest and that the type ' 
of passages and questionsTwould be similar. The differences in the control of reading time 
(per passage rather than a fixed time for the entire test) and th^* sequencing of questions 
and passages were explained. Dyring the test, the participants turned to successive pages 
on cue from the experimenter. Ten seconds per question were alldwed for previewing. 
Thus, for example^hen there were four questions on a passage, the. question page was 
presentedrfor 40 seconds. After reading, the question^page \yas presented for 15 seconds- 
per question. The extVa tinje was to allow for marking the answer on a machine -scorable 
aoswer sheet. Passages were presented to allow, for a single reading at a rate of 100 
words per minute. 

Results - 

-Pretest scores were used as the ability Measure. 9 Participants were dividedrirrfo high 
and low ability groups based on the overall pretest median scoVe of 15.5 (an RQL of 10.3 
on the test norms). The sample size in the cells of the resulting 2x2x2 matrix ranged from 
17 to 37. The pretest scores were analyzed using an unweighted means, analysis of' 
variance (A NOVA) to ''determine If random assignment of participants to the^ various * 
passage revision conditions resulted in groups of equivalent reading skills. ' 

The only source of variance to reach significance was the main*effect of ability (F = 
39.0, df = 1*222;. p < .001), simply reflecting the intentional high/low ability grouping. 
Thus, the ability' levels across each 'text condition were equal prior to experimental 
treatment. * ^ 

K to • 

^m* The total number of correct answens to the questions on the experimental passages 
was evaluated using a 2x2x2 unweighted means ANOVA. The only significant effects- were 
the main effects due to ability (F (1,122) = 10ll8, p < .001) and vocabulary (F (Jh,222) = 5.2, . 
p < .02). The main effect of ability, which was expected, dimply indicated that high 
ability read*j|s comprehend ^nc^e than do* low ability readers. The vocabulary effect, 
while statistically significant, was of only minor practical significance. The mean percent 
correct without vocabulary simplification was ,60.0. Simplifying vocabulary only raised 
performance by three percentage ptfints--to 63.3 percent. 

The results, summarized across ability levels, are presented Jn Figure 1. Most 
surprising is the lack of any .significant interactions. In particular, an interaction of the 
readability variables with, reader ability was expected. Since the high ability readers all ' 
scored atTove 10.3^RGL op the Nelson-Denny test, they were reasonably go<yi compre- ' 
henders. As Klare (1976) reports, readability manipulations are not as effective for 
"good" readers. However, since the low ability readers had a median pretest RGL score of 
8.7, these readability manipulations would surely affect their performance. YeJ, for the 
low ability group, only one percentage point separates Conditions O and V. Performance 
under Condition S-V is even slightly worse than performance on the original passages. 
Yet, according to the Kincaid-Flesch formula (Kincaid et al., 1975), the passages hajf an 
average RGL of £.5 compared to 11.5 for the original. This is the condition Ci.e., the joint 
manipulation of vocabulary and sentence) that Klare (1976) predicted would have the * 
maximum effect of on comprehension. 
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Legend 

V = Vocabulary simplified 

S = Sentences simpl^ied 

S*V = Sentences and 

* vocabulary simplified 



Figure 1. Rercent correct under each revision 
* 1 condition, Experiment 1. 



The results of this experiment offer no support for the hypothesis that readability 
formulas can be used as guides for simplifying text. Simplifying vocabulary and sentences^ 
has little if any effeqt on performance even though the readability, according to formula, 
is greatly improved. Even the- small jeffects obtained are contrary %o the predictions 
arising from the readability scores in Table 1. That is, white simplifying complex 
sentences produced the greatest improvement in t readability, score, yfcfie vocabulary 
simplification is what produced the positive, though trivial, performance effects. Even 
this effect is lost when the sentences are also simplified. 



***** 




EXPERIMENT 2 



In Experiment 1, the passages were presented to allow for a reading rate of 100 words 
per minute. It was hypothesized that the absence of a speed component^ might be the 
factor accounting for the lack of a strong effect of increased readability on comprehen- 
sion. Basically, given enough time most readers can struggle through a passage regardless 
of how difficult it is. Indeed, Klare (1976) reports that leisurely reading time is a primary 
, factor in negating readability effects. 

/ , 

In this experiment, the reading-to-do task approach (i.e., a minimal rpembry 
requirement) was maintained but the speed factor was increased. This .was accomplished 
by presenting the passages under the standard instructions of the. Nelson-Denny reading 
test, which,' like most reading ^£sts, follows a reading-to-do approach.. Questions and 
passages are pf&sented together and the reader must find the answers to the questions. 
The Nelson-Denny has the additional characteristic' of being a highly speeded test, 
allowing just over half of the test time (35 minutes) used in Experiment 1. Thus, following 
the. test procedures provides a speeded reading-to-do task that should maximize the 
effects of the readability manipulations. Indeed, all factors identified in Kjarels (1?7<J) 
model are fitfSd to produce maximum readability effects. 
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Method 



A total of 57-1 recruits were tested, with 143 to 146 randomly assigned to each 
condition. Afterprete-sting,, participants received one of the experimental sets of 
booklets. In each>ooklet, a passage and its associated questions were onjfacing pages. 
Participants were given the published instructions for the Nelson-Denny test' except that 
,they were required .to mark their answers on a machine-scorable^answer sheet. Under the 
instructions, the participants' are encouraged to move through the passages as quickly as 
possible, but they are free to skip passages,or return to earlier passages. *' ' 

■ V • • 

* Results and Discussion * « • 

r » . * • 

Recruits were divided into high, medium, and low ability groups based on the overall 
distribution of pretest scores. The low ability group consisted of those with scores below 
10.0 (7.9 RGL)?'the high ability group, with scores above 16.0 (10-7 RGL); and the medium 
ability group, with scores between 10.0 and 16.0. Because ttfe levels of the ability 
variable were defined in terms of the' overall distribution, sample size across the 12 cells 
of the experiment was variable, ranging from 34 to 61. " JF* 

■ • j V ' 

Pretest scores were analyzed using a 3x2x2 unweighted means ANOVA". The main 
eTfect due to ability was. highly significant (F(2,559) = 270.6, p < .001). The main effect 
of sentences was also significant (F(l,559) = 6.6, p < .01), indicating that the random 
assignment to conditions was not effective. An examination of the data indicates that 
"ft^gfj^ggi" Conditions^ and S-V was lower than that in Conditions O and V. Further, 
while thefflali^^ffect is significant, Jhe. primary difference appears to be with the high, 
ability group. The mean percent , cot rect on the pretest for the sentence original and 
sentence simplified conditions was 37.2 and 37.5 percent respectively for medium ability 
participants, compared-to 22.5 arid 21.5 percent for low ability participants. 

Interest was primarily on the -medium and. low ability readers since this is where the - 
readability effects are expected. Since the bias in assignment seems to be primarily 'in • 
the high ability groups, the pretest data were analyzed again using only the medium and 
low ability participants. The 2x2x2 unweighted means ANOVA» yielded ability as the only 
significant effect (F; (1,376) = 850.8, p^l). The sentence* effect did not' even approach 
significance (F < 1.0). X • 

Analysis of the experimental effects was carried out using only data from medium 
and Jow ability participants. 2 A 2x2x2 unweighted mjeanj^NOVA of the total number. of 
items coprect yielded significant main*effects due to ability (F(l,376) = 109.4, p < .001) 
and vocabulary (F(l,376) = 5.6, p < .02). In addition, the vocabulary'-by-ability interaction 
approached significance (F(l,376) = 3.23, p< .08). - The interaction is in:;the right direction; 
the vocabulary effect is stronger with the low abilky'participants. However, the'effect is s 
minpr, with the mean pfercent correct under conditions of vocabulary Unchanged versus 
vocabulary simplified biing 26.0 and 30.7. percent respectively for the low' ability 
condition. This is, in fact, similar to' the magnitude of the vocabulary effect in 
Experiment!. 3 



: 2 Analyses of covariance with ability as the covariate, which were carried on the 
entire data set in parallel to the ANOVAs, led to identical conclusions. 

3 An additional analysis carried out on the proportion of items correct to those 
attempted produced similar results. , < 
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The findings are displayed in Figure 2 with each revision Condition contrasted to 
performance with the original passages. Again, the findings look very similar to those 
obtained in Experiment 1. Condition S yielded the poorest performance, though not* 
significant, for both ability levels. Condition V facilitated performance only for 'low 
ability participants, but performance was increased by only five percentage points. 
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4 Figure 2. Percent correct under each revision condition for 

• medium and low ability groups, Experiment 2. * 



Increasing ^the speed of the test sh6uld have increased the effects of vocabulary and 
sentence simplification (Klare*, 1976). The test certainly was highly speeded; 50 percent 
of the participants did not respond to any of the last 1 ten items and only 1 1 percent 
answered the last itenri. Yet, .there is no practical effect of the more readable writing. 
An additional analysis was performed to determine, if perhaps increased readability 
resulted in a* lower error rate; that is, with the "difficult" test, readers could have given 
up and raced through the test primarily guessing. Under this strategy, they would 
increase the absolute number of items answered correctly but not the proportion correct. 
Thus, a 2x2x2 unweighted means ANOyA was carried out on the number correct relative . 
to the number of items attempted. ;As with the analysis 'of the total number of Correct 
items, the main effects of ability and vocabulary achieved significance (F( 1*376) = 28-9, p 
< .001 and F(l,376) = 8.7, p*< .01, respectively).* None of .the interactions achieved 
significance. As with the analysis of total correct, the vocabulary effect is of little 
pfactical significance, with only a difference of two percentage points between the means 
for the main "effect '0.58 for vocabulary simplified and .56 for vocabulary original). 

As with Experiment 1, the present results pffer na support to the hypothesis that the 
procedures of simplifying vocabulary and sentences by themselves will improve the 
comprehensibility of text materials. In this study, as with the previous one, the task and- 
testing conditions were v not especially motivating. In addition in this study; the test 
conditipns were highly speeded: According to Klare (1976), the-. combination of low 
motivation, a highly speeded test, and low ability levels should maximize the likelihood 
that at least Condition S-V would produce a meaningful improvement in performance. For 
low ability participants, the materials were simplified from about three grade levels 
above their average reading skill* to about two grade levels below their average reading 
skill. Yet, this large manipulation produced an effect that must be considered to be of no 
practical significance. • 



4 EXPERIMENT 3 

In Experiments 1 and 2, the test of comprehension was the ability to answer multiple- 
choice^yestions about the passages! It was hypothesized that 'the failure to find effects 
of ? practical significance 'may have been due to the use of the multiple-choice questions. 
The Nelson-Denny questions and procedures which are designed to distinguish between 
people with differing levels of reading skill, are apparently quite successful ifP doing so 
(Buros, 1968). However/ recent research findings indicate that many 'questions on reading 
comprehension. tests can be answered well above the chance level without the relevant 
.passage being read (Tuinman, 1973-197^ Pyrczak, 1972). Thus, it would seem tljaj many • 
reading^e*ts can successfully distinguish levels of reading skill with 'items that do not 
always assess reading skill. These items must be measuring individual difference variables 
that generally are highly correlated with reading skill. 

t 

Itefns that measure variables correlated with reading skill may be. acceptable for 
assessing individual differences irv skill. However, in* Experiments 1 and 2, the intent was 
to meastire differences in passages, not individuals. Here, questions that do not depend on 
reading the passage for a correct answer are clearly not acceptable. Comprehension 
differences due to difference in passage readability ^cannot be expected if the passages 
need not be read. • 

In this experiment, comprehension whs tested by a cloze comprehension test (Taylor, 
1953). The passages were presented to the participant with every fifth word deleted and 
the task was to fill in the deleted* words. This Is the procedure commonly used to assess 
Comprehension in the contemporary development of readability? formulas (Kincaid e\§l. $ 
1975; Caylor, Sticht, Fox, & For.d, 1973). Once again, the experimental conditions were 
optimal for obtaining nhe predicted effects due to readability revisions (vocabulary andi 
\sentence simplification). • ' 

* y 
Method * * 

Only passages two and three of the Nelsoh-Denny test were used in this experiment. 
The mean Kincaid-Flesch (Kincaid et al., 1975) readability scores for these passages were 
12.3, Kk9,'*6.0, and 5.7 RGLs for Conditions O, V, S, and V-S respectively. Thus, thfe 
distribution of readability scores was about the same here as' for the entire set^of 
passages. The standard cloze procedure (Taylor, 1.953), where every fifth word is deleted* 
was utilized.. Each passage in each condition was prepared with five different cloze^ 
deletion patterns. That is, in one version, the 1st, 6th, 11th, etc. words-were deleted, 
while in the next version, the 2nd, 7th, 12th, etc. words were deleted. This progression 
continued across all five versions until all words had been deleted. \ . ^ 

The passages were presented on successive pages of a booklet. Participants were told 
to move to the second passage after completing the first. No time limit was imposed for 
completion, which is consistent with standard cloze testing procedures. In all, .124 
recruits were tested, with 38 assigned to each condition. 
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Results * 

■ 1 mm \ a- * 

Recruits were divided into high and low ability groups based on the overall median . 
pretest score. The pretesrt performance was"* then analyzed with a 2x2x2 unweighted 
means ANOVA f Once again the results of the analysis indicated that' the* random 
assignment to groups did not!resulj in equivalent groups : within ability levels. In addition 
to the expected significance of ability (F (1,166) = 229.0, p < .01), the ability-by-sentence 
and the sentence-by-vocabul^ry interactions we^e also significant .(F (1,116) = 4.2 and 7.9 
respectively, p<.05). Exanrunatipn of the means for the pretest shows that the significant 
interactions are due to group differences of ufr to seven percentage points at both ability 
lefvels. Thus, the attempt to assign participants randomly to readability conditions was 
not successful, and ai^ subsequent ANOVA'would have been uninterpretable* Instead, the 
experiment^ data \rere analyzed, by an analysis of covariance with the pretestASCoYe 
serving as the, covariate. The cloze performance was scored using both a stringent 
peritenon and a lenient criterion. Under the stringent-criterion, the blank had to be filled 
' in with the exact word that had been deleted. Under the lenient criterion, synonyms were 
allowed, i Two scorers were used and both had to agree that a fespoRs£*was a synonym. 
The cloze scores averaged across passages served, as the basic data for 'analysis. T 

The analysis of covariance of exact cloze performance failed 0 to yield any effects 
„ significant^ the .05 level. However,"the vocabulary-by-sentefice? interaction approached 
significance (F(l,119) = 3.09, p<.10). The analysis of covariance of synonym cloze" scores 
yielded virtually identical results, with the vocabulary-by-sentence interaction once again 
approaching significance (F(l,119) = 3.06,, p<.ltJ) ; The adjusted means under the exact 
~scc>ring criterion are presented in Figure 3, where it can b£ seen tffat-the interaction is 
due to a slight decrease in performance when only vocabulary or sentiences are simplified, 
but a slight Increment in performance when both are simplified The difference between 
Conditions O and S-V is only 1.5 percentage points. Once,again, minimal effects due to 
simplification were obtained. The'mean cloze scopes are 'generally at the level considered 
to reflect an adequate level of comprehension (i.e.y 38% cloZe) (Bormuth, 1967; Rankin* & 
Culhane', 1969). thus, neither a basement nor a ceiling e'ffect limited the range oi the 
means. It would 'seem thep that, under most conditions, revising materials by simplifying 
vocabulary and sentences is not an effective way of improving comprehension, at least 
* when, the reader is not required to organize .and store the text. 
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Figure -3* Proportion correct (adjusted means) under exact cloze 
scoring for each revision condition, Experiment 3.' 



XPERIMENT H 



In all three of the previous experiments, simplifying vocabulary and sentences failed 
to facilitate comprehension. This finding seems to contradict a»long history- of learning 
research in which it is- a well established fact, that frequently use* and shorter sentences 
are easier" than less frequently used words and longer sentences.. However, the word • 
frequency and sentence complexity factors are most strongly established ^paradigms ' 
involving repeated learning trials and retention intervals. 4 . Indeed, the research findings 
on word and sentence variables, referenced in -readable writing recommendations (e.g., > 
Klare, 1975) come from studies of learning. It may be that these variables aYe relevant to .' 
memory, but not Comprehension, or at least comprehension when memory is not required. 

• Without a memory requirement, there is only a minimal comprehension* requirement. 
As Carroll (1972, p .7) has Suggested: . there |s little use in comprehending a-message 
unless the outcome'of ,that comprehension is remembered arid "t^ans'fer'ed to long term 
memory." Perhaps this statement should be , tempered to suggest that- .a memory v 
requirement will typically necessitate greater. Comprehension in "the sense of deeper 'or^ 
more elaborate processing of the information. . > , 



•Fass and "Schumacher (1978) found that simplification of text facilitated comprehen- , 
sion when the readers did hot know what information in the tex%had to be learned (i.e., 
memory Was required). However, the simplification effects were, reduced or eliminated 
by motivating or instructing subjects" to engage in deeper, processing activities, ^hus, it v 
may be that simplification in some way compensates for, or facilitates, the processing 
activities when required. In the first three experiments, however, .there was no demand to - 
organize, integrate,- or store the information in the passage'. In the cloze- test, isolated 
phrases are studied. In the multiple-choice testing, the readers can skim the passage* 
superficially looking for signals (e.g., key words) 46 direct them -to the answer to the 
question. Thus, simplification may not have Been effective simply, because there were 
minimal processing demands. . . ' • 

In this experiment, the same materials were used arid a merriory component was 
added to" the task in an attempt to : increase the requirement for processing activities. 
Basically, in this experiment, a reading-to-learri task (Sticht et al., J 977) similar to the' 
.situation faced „ in the classroom was employed. Each passage was .read with the . 
participant "hot knowing the comprehension questions. Thus, "comprehending" the,passage 
no'w meant acquiring all of the information presented,, ' ' \ • ' ' * f . " • 

Method-, * * /»- 

~7~ - ' ' ■ ) • 

- t - A total of recruits were .tested! with *60 to 62 assigned to' each of the four 
revision conditions. After, pretesting, participants received W of the .experimental 



booklets; The sequencing of materials In the booklets, the presentation procures, and 
the time allowed for passage and questions were identical to those in Experiment 1, 
except that the questions on a passage did not precede the passage. 



Of course, variables like "word frequency are rela/ted to the speed of word./ 
recognition, but this effect .may not be relevant in the context of text reading. . First, 'the 
effect is measured in milliseconds and hence may not have a practical effect. Second, 
the effect seems to be, dependent on the absence of an/ context for the words to' be 
identified (Broadberit, 1.967). , ' 
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Results and Discussion 



Paticipants were divided into high and low abiiky Subgroups based oft the median 
score of 15.3 (10.2 RGL) for the overall distribution of pretest scores. The pretest scopes**' 
\/ere subjected to a 2x2x2 unweighted means ANOVA to determine if the. assumption of 
'random assignment ^to readability condrtions had been satisfied. The only factor to* 
achieve significance in the analysis; was the main effect clue to ability (F (1,236) = 421.3, 
p< ;001), which simply reflected the intended division of readers into ability groups. 

- The total nunrtber of correct items on the experimental passages was analyzed using a 
2x2x2 unweighted? means ANOVA. The main effects due to ability and vocabulary reached 
significance .(F(ty236) = 126.1^p<.00J and F(l,236) = 9.7,><.002, respectively). Tf\e 
interaction of vocabulary-by-^bility also achieved significance (F(i;236) = 6.3, p<.0l). - 
The percent correct under each 'revision condition is shown separately in "Figure 4 for high 
and low -ability. participants. The results for' the high ability re'aders look virtually 
identical *to the findings in the^previous -three experiments; there is virtually no effect 
when ^vocabulSry is simplified and a slight decrease in performance when* sentences are t 
simplified. , - ' * * 

. The performance of low abiflty readers, in contrast to the previous findings, shows a 
strong effect due to vocabulary simplification --a 13-percent increase, in performance. 
While no* significant, the sentence manipulation also resulted^ for the first time 'in an 
improvement in performance. Thus, for low ability readers,. simplification seems to.be 
effective when a memory . requirement is present. While the effects are significant, 
however, the findings are still not consistent with readability predictions. The vocabulary 
manipulation resulted in theymallest change in Readability, but the largest effect on the 
comprehension .test.' The^mbination of vocabulary and sentence simplification is 
additive in its effects on readability formOla scores (i.e., maximum readability improve- ' 
ment is obtained under Cbndijion S-V). However, in eacfr of the 'experiments, the effect. 
_9n comprehension f (or memory) performance seems to be an average of the individual 
effects (i.e., performance under Condition S-V falls between Conditions S and V). Thus, 
the conclusion must be that/ even under the limited conditions where readability revisions 
result in improved performance, the formula score jcan^iot be relied on to provide even a x 
ranking of the relative difficulty'of the revised material. ' ^ < 
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Percent correct under each revision conditiort for 
high and low ability groups, Experiment 4. 



EXPERIMENT 5 



Method 



The 'purpose of this experiment was twofold First, the correlational relationship of 
readability and comprehension was contrasted to the causative (or manipulated) effects of 

- readability on comprehension. Second, the adequacy of the Nelson-Qenny comprehension 
questions for reflecting readability effects was assessed. Results from the previous 
experiments suggest that revisions for readability are only effective when the comprehen- 
sion test has a memory component. This, in turn, implies that the sentence and 
•vocabulary variables 1 are^correlative but not causative variables in reading comprehension. . 
That is, although simple'&ntences and simple vocabulary are characteristic of easy-tq- 
comprehend ma^rial, they are not the factors underlying improved comprehension when 
no memory requirement is imposed? The relationship of readability and comprehension 
c^n be assessed by examining the correlation between the variables within experimental 

* conditions in contrast to the* causative effect that was assessed by comparing perfor- 
mance across conditions*/The between-groups data faif to support a causative effect of 

"^sentence and word complexity on reading comprehension. 

In this experiment, the same data wpe reexamined to determine^ readability and 
comprehension covary within groups. This^ssfessment served as a cheeky on the adequacy 
of the Nelspn-E^ffiny test questions as measure of comprehension.. As discussed in 
Experiment 3, the cloze testing has already supported the conclusions of Experiments 1 
and 2. Howevei^ since .the support is for the null hypothesis, there is still no assurance- 
that the it&ms were adequate for reflecting readability differences. An effect dife % to 
vocabulary simplification was obtained in Experiment 4. However, to* the extent that the 
\ multiple-choice question^ were passage independent (could be correctly answered without 
reading the passage), the potential magnitude oflfiis effect, as well as a' potential 
sentence- effect, was suppressed. TJius, even in this experiment, the adequacy of the test 
items is not assured. However, if the questions are inadequate for reflecting between- 
group differences (i.e., they are passage independent), thenjlhdy will also be inadequate 
for reflecting within-condition differences in readability and no correlation between the 
readability and comprehension scores for the eight passages would be expected. ^ 

*■ * 

The data frorcu-Experiments 1 and 4, the two experiments* in which all participants 
answered all questions, were used for the^se analyses. Readability scores on each passage, 
calculated using the Kincaid-Fle^ch* formdla, (Kincaid et al., 1975) servedks one of the 
variables. The mean number of correctly -answered items on a passage ma particular 
'readability condition served v as the second .variable. Since there were eight questions on 
the first passage, the a^ean score on this pkssage was halved for the analysis. 

' v 

Results and Discussion 

The Kincaid-Flesch readability score for eachN)f the eight passages under eacjK 
revision condition is shown in Table 2. Each set of eigh/readaibility scores was used in the 
calculation of^two Pearson product moment correlation C9efficients, one using^s the 
second variable, the appropriate participants from Experiment 1; &nd the other, the 
participants from Experiment 4. . 

The resulting correlation coefficients shown in Table 3, were quite high except under 
Condition V--where orjly vocabulary was simplified. Thus, the data -strongly support the 
adequacy of the Nelson- Denn^ test questions for reflecting differences in the readability 
of the passages. More correct answers were given* on passages with easier readability 
scores.- _ . - 
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Table 2 



Kincaid-Flesch Readability. Scores for Each 
Nelson-Denny Passage After Each Revisioa 



Revision Condition 



Passage 
• 


Original 

(o)j v ' 


Vocabulary 
Simplified 
/ (V) - . 


Sentences 
Simplified 
(S) • 


Sentences- & 
Vocabulary 
. Simplified 
(S-V) . 


1 


8^5 


7.2 


4.2 


3.4 » 




9.4 


^ 8.2 


» - 6.1 


' 4.5 


3 


12.3 


. 11.2 




• . 3.8 • 


tt 


13'. 3 


s 15.4 ' 


. ri .2 


8'. 2 


5 


- 8.2 w 


6.7 / 


6.3 


* .5.2 


■$ 


s f '3 . 2 


- 9.4 


8.1^ 


6.5 


7 


10.4 


7.6 


' ' 8.7 - 


5.8 


8 


16.6 


14.8 • 

" L. 


7.9 


6.6. 


t 


• • 


\ 
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Correlation Between Passage Re^clability and Comprehension 
For Each Revision Condition in Experiments 1 and k 



Revision Condition 



1 



Experiment 



Original (O) 

Vocabulary simplified XV) * 
'Sentences simplified (S) . 
Sentences and vocabulary Simplified (S-V) 



-.704 
-.20 
-."85* 
'-.77* 



.57 
.07 
.68* 
.84* 



Note . N = 8 (the 8 passages) for "each Pearson product'moment correlation. 
*p < .05. . 
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More< importantly, these data add substance to the hypothesis <that word and sentence 
difficulty are correlative, but not causative, factors in comprehension*. The readability, 
formula that basically - measures word and sentence difficulty was predictive of compre-' 
hension in three of four conditions irf two experiments. Yet ; 'the manipulation .of • 
difficulty did not improve 'comprehension-- except for vocabulary .simplification when* 
memory is required of low ability readers. V < \ 

The findings from this study are not consis^ht with, a similar analysis carried out by, 
Entin and Klare (1*978) using the passages from an alternati^'form 6f the Nelson-Denny 
test*. They found a correlation coefficient of 'only TlO between performance on multiple- 
choice test items 3nd passage readability. When the scores were .corrected for pasj&ge . 
independence, the correlation value increased to M. Perhaps Entin and Klare obtained 
such low values Because they allowed extensive reacfing tim'e^even longer than in x ffiese ' 
experiments, artd their readers -were university students (i.e., high ability readers)^ Entin 
ancf Klare's second analysis, correcting for passage independence, suggests thai a similar 
procedure carried out on the present data, would result "in even higher correlation 
coefficients between readability and comprehension. Yet, the results from Experiment 3,, 
In which H cloze test: was used, suggest that we still would not obtain effects due to the 
manipulation of readability. ■ 

• ' • ■ f / 

DISCUSSION AND CONCLUSIONS 



x In this series of experiments, the readability of * passages was manipulated over, wide 
- \ .ranges from an RGL of il.p to 5,5, by following the most basic.and fundamental rules for - 
\ readable writing. The syntactic structure of sentences was simplified and more familiar . 
words ^ere substituted. Yet, except for one instance, these manipulations had no 
practicareffect on comprehension, regardless of the reading skill of the participants. The 
experimental conditions met every requirement of Klare's (1-976) model for producing 
t comprehension effects. In one experiment (Experiment the conditions of the develop- 
ment of the military readability formulas (Kincaid et 3^197^. Cayldfeet ah, 1973), in the 
jnain„were replicated. Yet, even under these cireumstances,^eadabifity manipulation did 
not-affect comprehension. - ' * * 

> ^\ * ' ' \ < ^ * 

Inadequate test questions cannot account for the findings since two test types 
• (multiple-choice and cloze) failed to yield * differences. The most suspect test 
type— multiple-choice--was the type with which significant comprehension effects were 
eventually obtained. In additioh, performance on the multiple-choice items w.as strongly 
related to naturally • occurring differences in readability. Thus, the .questions were 
* adequate for measuring readability effects. * , 

The only instance in which the manipulations in readability had a practical effect on' 
comprehension was wheri the comprehension auesions were not known at the tii^ie of 
reading (i7e., in the reading-to-learn format). Even here, the order of effect on 
comprehenson was not consistent with the order of readability differences. The smallest * 
change in readability resulted in the greatest comprehension effect. However, th<* finding \ 
that simplification facilitates comprehension only on reading-to-le^rn tasks is consistent , 
with tKe findings of Siegel et al. (199**)., In that experiment, both vocabulary and 
sentences in technical materials \yere simplified. 'The materials were then used either as 
„ a correspondence, training text, with a test after studying, or as a reference manual, with ' 
. questions asked while the manual was available. \\ was only in the training, situation—and 
then only when the manual was not supplemented by other instruction— that the effects" 
£ue to simplification ,<vere obtained. Thejact that simplification facilitates performance 
in reading-to-learn but not in reading-to-do indicates that the 'effect is fiot due simply to 
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improved comprehension of the individual units. Simplifying vocabulary does not improve 
performance simply by improving 'word knowledge. Simplifying sentences do^s not 
improve performance simply by .improving comprehension of single sentences. If the 
effects were<on isolated \fords and sentences, then clearly performance on t^e cloze test 
and in finding^ the answers to multiple-choice questions without a memory requirement 
would have been facilitated. - . V 

Th? requirements in the two types of tasks must be compared to determine the basis 
of the simplication effect. In reading-to-learn, the reader does not know what 
information must be retained to demonstrate comprehension. Thus, the" material mCist be 
read with the goal of remembering everything that the reader judges to be important. 
This will most likely far exceed immediate memory capacity. Therefore, the task 
demands that the information in the passages be organized, integrated, and stored .even 
for an immediate memory task. In reading-to r do, integration and organization typically 
are. not required. Rather, the relevance of information is judged as reading progresses and 
can be "discarded if it is not relevant to any of the specific goals. Thus, when simplifying 
vocabulary and sentences does facilitateScomprehension, it appears to be through the 
organization and integration of the text. Perhaps the simplification resylts in more 
semantic associates and smaller chunks of information and, therefore, allows for greater 
flexibility in (ttganizing and integrating the information. This in turn may result in storing 
of more information or K in more efficient retrieval for answering the questions. 

Fass and Schumacher (1978) found that instructing subjects to engage in deeper 
processing activities reduces the effects- of simplifying text in a reading-to-learn 
situation. These findings, which are consistent with tbe hypothesis of this research, 
further suggest that providing guidance on how to process the text or motivation to carry 
out cleeper processing can assist the reader in overcoming (i.e., comprehending) a difficult 
text, as well as simplifying the text for the reader. 

In sum, text simplification "seems to be one means, of meeting the requirements for 
text processing. Thus, simplification should facilitate comprehension to the extent that 
deeper processing (i.e., integration and organization of the text) is "required. This would 
include any reading-to-learn an<lreading r to-do tasks in which the reader must draw an 
inference, summarize the information, or perhaps carry away in memory a large string of 
'facts (e.g., a procedure^ Conversely, the effects of, or need for, simplification can be 
reduced by reducing, the* organizational demands. This could be done by providing an 
organizational structure, by motivating or instructing subjects to engage in appropriate 
processing activities, or by ensuring that the reader is knowledgeable in the content area. 
Klare (1976), for example, reports that,. if the reader has knowledge of the content area, 
the 17 effects of simplifying will be negated. Basically, content knowledge provides a 
schema (Anderson/ Spiro, <5c, Montague, 1977) or organizational structure for the 
information. A new structure or integration is not required. 

The practical implication of the present findings is that a readability formula is not 
an effective production criterion, even when 'the writer does not deliberately write to the 
formula. Simplifications based on readable writing guidelines will be ineffective for 
technical manuals, at least where finding factual information' is the task. For training 
manuals used in classroom instruction or in studying for advancement exams and for 
technical manuals read for "understanding," readable writing revision may have some 
effect under limited circumstances, but the effect will not be of the magnitude predicted 
by the readability formula score. Not only will formulas overstate the ^magnitude of a 
change, but they^ will not even effectively Yank the difficulty of 'texts rewritten according 
to readable writing rules. \ ' 



Of course, revisions based on readable writing guidelines can be effective at the 
extreme levels of diffiuclty. If the reader, "has no knowledge of the meaning of a 
significant proportion of the vocabulary, the sentences are extremely complex, and the 
reading task is more than to "look up," then a readability formula score can be an 
effective criterion for improving comprehension. That is, if the writer does not write to 
the formula. • — 

In addition to being ineffective, the use of a readability formula seems to have 
limited the consideration of. comprehension variables. Mjfre than just sentence and word 
factors determine comprehensibility, especially in tejsfinical manuals. Yet, as discussed 
earlier, comprehensibility specifications are presently limited to readability. Although 
graphics and the^coordination of graphics and text play integral roles in technical 
manuals, readability analyses totally miss these factors. Procedural -listing vs. paragraph 
presentation of information, highlighting techniques, and the organization of information 
within a paragraph are all variables affecting the comprehensibility of text that are 
independent of readability. > 

, How are all of the comprehensibility factors to be considered in the production of a 
manual? There are three alternatives: ' guidelines, regulations, and changing the 
production system. Guidelines would not appear to be an effective approach. Although 
many such guidelines are quite Jogical (e\g., place the text and the relevant graphic on the 
same pr facing pages), they are violated constantly. While there are innumerable books 
and training courses providing guidance for technical writers, comprehensibility is a 
continuing problem. Thus, guidance alone has proyen ineffective. The use of a readability 
formula as^a criterion is an attempt at objective regulation. 

An alternative to {he standard readability formula is a more complex formula that 
would include graphics, highlighting, and other diverse comprehensibility factors. Related 
to this notion is the recent -preparation of a military standard for comprehensible writing 
of technical manuals (Department of Defense, 1978). This standard attempts to translate 
all the relevant research on comprehension into concrete writing and design statements. 
For example, the number of graphics per page, the use of procedural statements, fnd the 
use of specific highlighting techniques are so explicitly described that this document could 
be used as a criterion or specification for writing and design. While a specification of this 
complexity might be effective in increasing comprehensibility, it probably would not be 
cost effectively itself. Writers and designers woukTneed training to use the specifica- 
tion and all details of the draft <pf the technical manual would have to be reviewed in* 
chelation to the explicit specifications. However, through a gradual evolution, including 
developing of training courses and programming the specifications into computer editing 
systems, a cost effective procedure for controlling the comprehensibility of manuals could 
be developed around such a standard. 

A similar but mote flexible system for comprehensibility control is embodied in 
McDonald-Ross and Waller's (1976) concept of a transformer in the 'text production 
process. The transformer is a grpup or individual whose, sole responsibility is to ensure 
that the text (or technical manual) is maximally usable for the intended -audience. e The 
transformer is competent in educational technology, editing, graphic design, arid the 
subject matter- area. The transformer, then, is responsible for ensuring that principles 
like those embodied in the comprehension standard (Department of Defense, 1*978) are 
applied appropriately. There should be a transformer function, or desk, in the production 
team and, when a contract is involved, the monitoring team. 
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The procurement of military hardware systems has traditionally encountered similar 
problems: The design process doe$ 0 not adequately attend' to thef manning requirements 
(i.e., the needs of the user). The Navy has recently included a transformer typte office in 
/ the hardware procurement system (Chief\^-t<Jav^l Operations, 1977). This office, whose 

acrdhym is HARDMAN, has the sole function of reviewing each phase of the procurement 
effort to erasure that the "people considerations" are fully addressed. It is only throogh 
\ a the institution of a complex specification or the institution of a transformer office 
analogous "to -the HARDMAN office that all aspects of a manual relevant to compre- 
hension can be controlled* 

/ ' <• 

* J RECOMMENDATION ' - 

y 9 The-text production process should be studied to determine how both government and 
contractor management of the production effort can be modified to increase attention to 
« Gomprehensibility. This should include a, consideration of computer-aided authoring 
systems, as well as changes in management priorities* apd in personnel qualifications. This 
* study provided no data that would justify the use of a readability formula as a means Qf 
controlling the comprehensibility of text. * 
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ORIGINAL AND THREE REVISIONS OF ONE OF THE EXPERIMENTAL - 
PASSAGES FROM THE NELSON-CENNY READING TEST 

Original • . 

The night was cloudy, .drizzling raih, which fell without intermission, added to 

the obscurity. Steadily, and^as noiselessly as possible, the Spaniards made their way along 
the main street, which had so lately resounded to theTfumult of battle. All was now 
hushed in silence; they were only reminded of the past by the occasional presence .of some 
solitary^borpse, or a dark heap of the slain, which too plainly told where the strife had 
been hottest. As they passed along the lanes and alleys which opeifed into 'the great 
-street, they easily fancied they discerned the shadowy forms of their foe lurking in^ 1 
ambush, ready to spring upon them. But it was only fancy; the city slept undisturbed even 
by the prolonged echoes of the tramp of^he horses, and the hoarse rumbling of the 
artillery and baggage trains. At length, a lighter space beyond the dusky line of buildings 
showed the van of the army {hat it was emerging on 5 the open causeway. They might well 
have congratulated, themselves on having thus escaped the dangers of an assault in the 
city itself, and that a brief time would place them in comparative safety on the opposite 
shore. 



Vocabulary Revised 



The night was cloudy^ and a sprinkling rain, which fell without a break, added to the 
darkness. Without stopping, and with as little noise as possible, the 'Spaniards made their 
way along the main street,' which- had so recently roared to the noise of battle. All was 
now hushed in silence; they were^pnly reminded of the past by the presence of a single # 
dead body, or a dark heap of the dead, which too clearly told where the battle had been 
worst. Asfthey passed along the fanes and alleys which opened into the great street, they 
easily fancied they saw the shadows of their enemy lying in wait, ready to spring upon 
them. Bifcjt was only fancy; the city slept without being bothered even by the constant 
sounds o/ the tramp of the horses, and the rough rumbling of_the cannons and baggage 
trains. At length, a bright space beyond the dark line of buildings showed the look-out^for 
the army that they were coming out on the open highway. Th£y might well have rejoiced 
on having^thus escaped -the- dangers of ar^attack in the city itself, and that ^ brief time, 
would place them in greater safety on the opposite shore. * ^ 

Sentences Revised * * 



The /night was cloudy. A drizzling rain added to the obscurity. If .fell without 
intermission. The Spaniards made their way along the main street. They moved steadily 

*and as' noiselessly as possible. The street had so, lately resounded to the tumult of battl£. 

\A11 was now hushed in silence. The occasional presence of some solitary corpse reminded , 
them of the past. A dark heap of $he slain was another reminder. Plainly, the strife had 
been hottest there. They passed along thfe lanes and alleys opening into the great' street. * 
They easily fancied the shadowy forays of their foe lurking in ambush. The enemy looked 
ready ^o spring upon them. But it Was only fancy. The city slept undistrubed by the 
hoarse rumbling of artillery and baggage trains. Even the prolonged echoes of the tramp 

^>f-hor-ses^did-not4disturix-the city. At length, there was a lighter space beyond the dusky 
line of buildings. This informed'the army van of their emergence onto the Qpen causeway. 
They might well ha$e congratulated themselves. They had thus*escaped the dangers of an 
assault in the city itself. A brief time would place them in comparative safety on the 
opposite shore. 
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Sentences and Vocabulary Revised - . 

* • < , . v 

The nigh Aras cloudy. A sprinkling rain added to the darkness. It fell without a ; 
break. The Spaniards made their way along the main street. They moved without 
stopping and with as tittle noise as possible. The street Kad so recently ^oar^d to the 
noise of battle. All was now hushed in silence. The presence of a single dead body 
reminded them of the past. A dark heap of the' dead also reminded them. Clearly, the* 
battle had been worst there. They passed along the lanes and alleys opening into the great 
street. They easily fancied the shadows of their enemy lying in, wait. The enemy locked 
ready tc* spring upon them. But it was only fancy. The city slept without being bothered 
by the rough rumbling of the cannons and baggage trains. Even-the constant sounds of the 
. tramp of horses did not bojher the city. At length, there was a bright space beyond the 
dark line of buildings. This informed the army look-out of their coming out onto the open 
highway. They might well have rejoiced. They had thus escaped the dangers of an attack 
m the city itself. A brief time would place *them>ih;greater safety on the opposite shore. " 
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